Multiple Kernel Machines Using Localized Kernels

Multiple kernel learning (Mkl) uses a convex combination of kernels where the weight of each kernel is optimized during training. However, Mkl assigns the same weight to a kernel over the whole input space. Localized multiple kernel learning (Lmkl) framework extends the Mkl framework to allow combining kernels with different weights in different regions of the input space by using a gating model. Lmkl extracts the relative importance of kernels in each region whereas Mkl gives their relative importance over the whole input space. In this paper, we generalize the Lmkl framework with a kernel-based gating model and derive the learning algorithm for binary classification. Empirical results on toy classification problems are used to illustrate the algorithm. Experiments on two bioinformatics data sets are performed to show that kernel machines can also be localized in a data-dependent way by using kernel values as gating model features. The localized variant achieves significantly higher accuracy on one of the bioinformatics data sets.